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Abstract 

Does engineering organizations knowledge use organizations 
and engineers knowledge workers? This paper says 
positively and discusses the need for knowledge management 
in engineering organizations. Unfortunately, the knowledge 
management literature does not provide much guidance on 
how to measure the success or benefits of doing knowledge 
management in an engineering or other knowledge using 
organization. This paper discusses research that proposes a 
definition of knowledge management success and 
dimensions and measures that organizations can use to 
value knowledge management success. A case study of a 
nuclear utility engineering organization is used to illustrate 
how these dimensions and measures can be used to 
demonstrate the success of a knowledge management 
initiative/project. 
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Introduction 

Knowledge workers defined by Davenport (2005) are 
those regarding knowledge as living and their main 
assets. Reinhardt et al. (2011) differentiated knowledge 
work from other forms of work through the primary 
task of "non-routine" problem , the solution to which 
requires a combination of convergent, divergent, 
and creative thinking. These definitions lead to the 
conclusion that engineers are knowledge workers 
performing knowledge work and engineering 
organizations are knowledge intensive organizations. 
Many knowledge intensive organizations utilize 
knowledge management (KM) to better manage their 
knowledge assets. Reasons to use KM come from a 
Jennex (2009) study on aging work forces who found 
that there is a great concern over the potential loss of 
knowledge due to retirement of engineers in the 
commercial nuclear industry. Also, Jennex (2006) 
reported NASA (National Aeronautic and Space 
Administration) due to the lack of KM practices has 
led to the loss of document based knowledge on how 



to build moon capable spacecraft. The observation is 
that engineering organizations need to successfully 
use KM to manage their knowledge. The research 
question investigated is that whetherthe Jennex- 
Smolnik-Croasdell definition of KM success can be 
confirmed when an engineering organization succeeds 
at KM or not. This is important as an organization 
must demonstrate when it does KM well. To answer 
this question, the paper presents a previously 
published case study of successful KM in a nuclear 
engineering organization and applies the Jennex- 
Smolnik-Croasdell measures of KM success to 
examine if it can be demonstrated that the KM 
program is successful. 

KM Success Background 

How to be successful with KM? Given the importance 
of knowledge to engineering organizations and the 
notable failure to capture knowledge as discussed 
above, it is important to answer this question. Initial 
research focused on the identification of critical 
success factors, CSFs, to initiate KM/KMS and the 
application of these CSFs to KM success models (Note: 
CSFs are areas in which satisfactory results ensure 
successful competitive performance and are the 
minimum key factors that an organization must have 
in order to achieve some goals (Alazami and Zairi, 
2003)). Davenport, DeLong, and Beers (1998) identified 
four objectives for knowledge-based projects: create 
knowledge repositories, improve knowledge access, 
enhance knowledge environments, and manage 
knowledge as an asset. KM projects are successful 
when there is a growth in resources attached to the 
project, a growth in knowledge content, a likelihood 
that a project would survive without the support of a 
particular individual, and some evidence of financial 
return (Davenport, DeLong, and Beers, 1998). Jennex 
and Olfman (2005) surveyed the literature on KM and 
KM project success to generate a list of KM CSFs: 

• A Knowledge Strategy that identifies users, 
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sources, processes, storage strategy, knowledge 
and links to knowledge for the KMS. 

• Motivation and Commitment of users including 
incentives and training 

• Integrated Technical Infrastructure including 
networks, databases/repositories, computers, 
software, KMS experts 

• An organizational culture and structure that 
supports learning and the sharing and use of 
knowledge 

• The wide knowledge structure of a common 
enterprise that is clearly articulated and easily 
understood 

• Senior Management support including allocation 
of resources, leadership, and providing training 

• Learning Organization 

• There is a clear goal and purpose for the KMS 

• Measures are established to assess the impacts 
of the KMS and the use of knowledge as well 
as verification that the right knowledge is 
captured 

• The search, retrieval, and visualization functions 
of the KMS support easy knowledge use 

• Work processes are designed that incorporate 
knowledge capture and use 

• Security/protection of knowledge 

Jennex and Olfman (2006) modified DeLone and 
McLean's (2003) IS Success Model to incorporate the 
KM CSFs listed above into a KM success model, see 
figure 1 (additionally Kundapur and Rodrigues, 2010, 
simulated the model to aid organizations in examining 
whether KM benefits will be found). This model uses 
KMS (system) quality, knowledge quality, and service 
quality as functional drivers for the use and impact of 
knowledge-based systems. KMS (system) quality 
refers to how well KMS perform with regard to its 
technical infrastructure and knowledge creation, 
storage, retrieval and application. Knowledge quality 
refers to the usefulness of knowledge artifacts in terms 
of their correctness and inclusion of contextual 
meaning as well as having appropriate linkages to 
knowledge. Service quality is a measurement of 
support for the systems in use. Performance impact is 
judged by the ability of these constructs to impart use 
of the systems and overall user satisfaction. 
Knowledge benefits are derived from the quality of 
the knowledge in the system and service dimensions 
associated with the system. Benefits are also a result of 
increased use and user satisfaction. 




Quality 

Sen/ice Quality 

FIG. 1 IENNEX AND OLFMAN (2006) KM SUCCESS MODEL 

Other literature identified similar factors that lead to 
success for KM related projects including flexible 
knowledge structures, knowledge-friendly culture, 
clear purpose and language, and multiple channels for 
knowledge transfer (Davenport, DeLong, and Beers, 
1998; Bhatti, Zaheer, and Rehman, 2011). Additionally, 
research into why KM can fail identified issues similar 
to Jennex and Olfman (2005) and include capturing the 
wrong or obsolete knowledge, a KM strategy that does 
not address organizational needs; lack or conflicting 
leadership, and lack of incentives for knowledge 
sharing (Lam and Chua, 2005; Keramati and Azadeh, 
2007; Weber, 2007; Jadoon and Hasnu, 2009; Bhatti, 
Zaheer, and Rehman, 2011). 

However, KM impacts are not well understood. 
Common identified impacts in KM projects involved 
operational improvements for a particular process or 
function. These were hard to measure leading to 
further research into what defined KM success and 
how to measure that success. Jennex, Smolnik, and 
Croasdell (2009) utilized a literature generated survey 
to propose a definition of KM success and then 
surveyed KM academics and practitioners to 
determine if the proposed dimensions and their 
associated measures of KM success are valid. The 
proposed definition of KM success is: 

"KM success is a multidimensional concept defined 
by capturing the right knowledge, getting the 
right knowledge to the right user, and using this 
knowledge to improve organizational and/or individual 
performance. KM success is measured by means of the 
dimensions: impact on business processes, strategy, 
leadership, and knowledge content." (Jennex, Smolnik, 
and Croasdell, 2009) 

Jennex, Smolnik, and Croasdell (2012) attempted to 
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validate the above KM success definition through 
construction and administration of a survey. The 
measures for the survey that are used to operationalize 
the dimensions have been identified in the literature, 
and then validated via an expert panel of fifteen 
experts who were given the items and asked to 
identify the dimension to which the item was 
associated and if more items were needed for each 
dimension. Results of the expert panel have been used 
to adjust the wording and placement of the items, and 
to clarify what each dimension meant. The final items 
(incorporating the measures) were used to generate 
the survey and were: 

Impact on Business Processes: 

1. My last KM project improved the efficiency of 
the supported processes 

2. My last KM project reduced costs for the 
supported business process 

3. My last KM project had a positive return on 
investment for the supported processes 

4. My last KM project improved the effectiveness 
of the supported processes. 

5. My last KM project improved decision making 
in the supported processes 

6. My last KM project improved resource 
allocation in the supported process 

Impact on KM Strategy 

1. My last KM project resulted in changes to my 
organization's KM goals 

2. My last KM project resulted in the creation 
or modification of knowledge related key 
performance indicators 

3. My last KM project resulted in changes to the 
way my organization assessed knowledge use 
in the organization 

4. My last KM project resulted in changes in my 
organization's incentives for using and sharing 
knowledge 

5. My last KM projected resulted in my 
organization increasing its awareness/mapping 
of knowledge sources and users 

6. My last KM projected resulted in increased 
resources for our KM systems and repositories 

7. My last KM project resulted in the creation of 
new or additional knowledge capture processes 

Leadership/Management Support 

1. My last KM project resulted in increased verbal/ 
political support for KM by top management 



2. My last KM project resulted in increased 
financial support for KM by top management 

3. My last KM project resulted in increased 
awareness of KM by top management 

4. My last KM project resulted in increased 
use/reliance on KM by top management 

Knowledge Content 

1. My last KM project resulted in increased 
knowledge content in our repositories 

2. My last KM project improved knowledge 
content quality of our repositories 

3. Increased use or intention to use of knowledge 
content as represented by the following two 
items: 

a. My last KM project resulted in my 
increased use or intention to use of 
knowledge content 

b. My last KM project resulted in others 
increased use or intention to use of 
knowledge content 

4. Increased identification of required knowledge 
and knowledge content sources as represented 
by the following two items: 

a. My last KM project resulted in my increased 
identification of required knowledge content 
and knowledge content sources 

b. My last KM project resulted in others 
increased identification of required knowledge 
content and knowledge content sources 

5. Increased demand and/or searching for 
knowledge content as represented by the 
following two items: 

a. My last KM project resulted in my 
increased demand and/or searching for 
knowledge content 

b. My last KM project resulted in others 
increased demand and/or searching for 
knowledge content 

Jennex, Smolnik, and Croasdell (2012) administered 
the survey to KM practitioners and scholars who were 
asked to respond based on their last KM project with 
the statement "your last KM initiative/project was 
considered successful" and they were then asked their 
agreement using a 7 point Likert scale on the above 25 
constructs. The 88 responses were divided into two 
analysis groups. Those who responded agree (6) or 
strongly agree (7) that their last KM project/initiative 
was considered successful (57 responses) were placed 
in the agree group while all other respondents (31 
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responses) were placed in the non-agree group. 
Respondents who responded slightly agree (5) to the 
success of their last KM project/initiative were placed 
in the non-agree group to help make the groups more 
equal in number and because it was felt that those 
who responded slightly agree may be biased against 
reporting their project/initiative as a failure. Dimensions 
were analyzed using three methods: the highest score 
for the associated items; the average of the scores for 
the associated items; and the total number of 
associated items met with an item score of 6 or 7 
needed to consider the item met. Scores were further 
analyzed to determine if the dimension was met for 
each response. Methods 1 and 2 determined whether 
the dimension was met and the score was greater than 
5. Method 3 considered the dimension met and if at 
least half of the items scores were greater than 5. 
Finally, responses were analyzed by determining how 
many dimensions and total items were satisfied. 
Means for each of these were generated for each group 
and t-tests were used to determine if the differences 
between groups were significant. 

TABLE 1 SURVEY RESULTS (AVG/(STD.DEV.)/(N) 



Dimen-sion 


High Value 
Method 


Average Value 
Method 


Item Count 
Method 




Agree 


Non- 
Agree 


Agree 


Non- 
Agree 


Agree 


Non- 
Agree 


Impact on 
Bus Process 


6.4 
(0.729) 

(57) 


5.6 
(1.31) 
(32) 


5.7 
(0.705) 
(57) 


4.7 
(1.06) 
(32) 


4.0 
(1.67) 
(57) 


2.1 

(1.64) 
(32) 


Impact on 

KM 
Strategy 


6.2 
(0.950) 
(57) 


5.8 
(0.898) 
(31) 


5.0 
(0.115) 
(57) 


4.5 
(0.891) 
(31) 


3.3 
(2.34) 
(57) 


2.0 
(1.84) 
(31) 


Leadership/ 
Mgmt 
Support 


6.0 
(1.05) 
(57) 


5.4 
(1.33) 
(31) 


5.2 
(1.20) 
(57) 


4.4 
(1.31) 
(31) 


1.9 
(1.49) 
(57) 


1.3 
(1.38) 
(31) 


Knowledge 
Content 


6.3 
(0.854) 
(53) 


5.8 
(1.08) 
(29) 


5.4 
(0.977) 
(53) 


4.7 
(1.12) 
(29) 


4.6 
(2.61) 
(53) 


2.6 
(2.23) 
(29) 



TABLE 2 GROUP COMPARISONS 



Dimensions and Items Met 
(Mean/(Std.Dev)) 


T-Test Comparison 
Between Groups: 
(Agree vs Non-Agree) 
Note all tests are 
significant 


Method 


Agree 
n=57 


Non-Agree 
n=32 


High Value 


3.4 
(0.9909) 


2.7 
(1.3102) 


t5i=2.61 


p < 0.01 


Average 
Value 


2.6 

(1.3595) 


1.3 
(1.2854) 


t 6i =4.26 


p < 0.01 


Item Count 


2.5 

(1.3379) 


1.4 
(1.3664) 


t«i=3.46 


p < 0.01 


Total Items 
(25 possible) 


13.4 
(6.71091) 


7.7 
(5.0902) 


t/9=4.57 


p < 0.01 



KM project/initiative measured items in more 
dimensions were. This suggested that the model of 
KM success is probably correct and that KM project/ 
initiative managers should use multiple measures in 
each of the four dimensions in order to measure 
success. A final analysis done was the splitting of 
the agreement group into agreement (41 responses 
answering 6 on project success) and strong agreement 
(16 responses answering 7 on project success) and t- 
tested to determine if the differences between these 
two groups were significant. This analysis yielded the 
most striking results as summarized in Table 3. 
Figures 2 and 3 visualize the findings from both 
analyses: 

TABLE 3 STRONGLY AGREE VS AGREE DIMENSIONS AND ITEMS MET, 
(MEAN/(STD.DEV)) 



Method 


Strongly Agree 
n=16 


Agree 
n=41 


t-test data 


High Value 


3.8 
(0.5439) 


3.2 
(1.0701) 


t 5 i=2.9787 
p<0,01 


Average 
Value 


3.5 
(1.0328) 


2.3 
(1.3233) 


t35=3.7243 
p<0,01 


Item Count 


3.4 
(0.8851) 


2.3 
(1.3398) 


t«=2.9997 
p<0,01 


Total Items 
(25 possible) 


17.4 
(6.1207) 


11.9 

(6.3332) 


t2s=3.0513 
p<0,01 



3.5 
3 

2.5 
2 

1.5 
1 

0.5 




□ 



n# Dimensions 
Met 



It 



FIG. 2 DIMENSIONS MET VS SUCCESS (7 MOST SUCCESSFUL, 4 
NOT SUCCESSFUL, 4 DIMENSIONS MAX) 



20 
15 
10 
5 



□# Items Met 



The results were mostly as expected: the more 
successful the KM project/initiative was, the more the 



7 6 5 4 

FIG. 3 ITEMS MET VS SUCCESS (7 MOST SUCCESSFUL, 4 NOT 
SUCCESSFUL, 25 ITEMS MAX) 

These figures show that the more successful a KM 
project/initiative is perceived to be, the more 
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dimensions from the KM success definition and the 
more item measures from this study are met. Why are 
there more dimensions and items met for successful 
KM projects over those less successful? KM does not 
exist in an organizational vacuum, and it neither can 
be bolted onto an organization nor can it be done 
independent of the organization. Knowledge use and 
value only occurs within the context of the knowledge 
users and the organization. To be successful with 
KM, organizations need to fully understand what 
knowledge is needed, who needs it, how it is used, 
and why it is used. Successful KM projects/initiatives 
look for knowledge use and value in a large variety of 
ways. Knowing how to measure the KM project/ 
initiative definitely helps and that is the value of this 
research. Knowledge use and, thus, KM must impact 
the processes that they support. Improved processes 
impact leadership/management, with the purpose to 
guide the performance of organization at its best. 
Improved knowledge use drives the KM organization 
to modify their KM strategy to reflect how it works. 
Finally, like any resource, the organization strives to 
accumulate useful knowledge. The KM success 
definition recognizes that KM success is in getting the 
right knowledge to the right people at the right time. 
The dimensions recognize that being successful with 
KM will be reflected in these definitions as discussed 
above. While having a useful list of measures , they 
will be better understood if put into context. The rest 
of this paper is about the application of these measures 
to reevaluate a case study of KM in an engineering 
organization. 

KM in an Engineering Organization 

Organizational Background 

The subject of engineering organization is part of a 
large, United States based, investor-owned utility. The 
utility, over 100 years old, with a service area of over 
50,000 square miles, provides electricity to over 11 
million people via 4.3 million residential and business 
accounts, with operating revenues of approximately 
$8.7 billion in 2002. The utility of net revenue has 
fluctuated wildly the last few years with a $2.1 billion 
loss in 2000, $2.4 billion in earnings in 2001 (primarily 
due to one time benefits from restructuring and other 
initiatives), and decreasing to $1.2 billion in earnings 
in 2002. To service its customers, the utility operates a 
transmission and distribution system and several large 
electrical generation plants and is organized into 3 
main line divisions, Transmission and Distribution, 
Power Generation, and Customer Service. Divisions 



such as Human Resources, Security, and Information 
Technology (IT) support the line divisions. The utility 
has approximately 12,500 employees. 

The Power Generation division is organized 
into operating units dedicated to supporting specific 
power generation sites. Each operating unit has 
line organizations such as Operations, Maintenance, 
Engineering, and Chemistry/Health Physics. Power 
Generation operating units are supported by dedicated 
units from the corporate support divisions (Security, 
Human Resources, IT). The engineering organization 
used for this case study is part of the nuclear operating 
unit of the Power Generation division and located at 
the largest electrical generation site operated by the 
utility. IT support is provided to this operating unit by 
Nuclear Information Systems (NIS) which 
administratively is part of the corporate IT division 
and operationally reports to both corporate IT and the 
nuclear unit of the Power Generation division. NIS 
supported engineering through its Engineering Support 
Systems group. This group consisting of a supervisor, 
two project manager/ analysts, and two developers 
was tasked with the maintenance of the eleven 
systems under NIS control. New systems or 
enhancements on existing systems were done at the 
instigation of engineering. Engineering through a 
charge back process paid costs associated with these 
projects and developers were hired as needed to 
support the work. 

At the time of the study, the engineering organization 
consisted of approximately 460 engineers disbursed 
among several different engineering groups reporting 
to the Station Technical, Nuclear Design Organization, 
Nuclear Oversight, and Procurement management 
structures. Industry restructuring has caused large 
drops in revenues driving the nuclear unit to reorganize 
engineering into a single organization consisting of 
330 engineers under the management of the Nuclear 
Design Organization. 

Basic KM Findings 

The organization is driven to capture and use 
knowledge. Since it is a nuclear plant, it falls under 
the guidance of the United States Nuclear Regulatory 
Commission (NRC). The NRC mandates that nuclear 
plants learn from events so that they are not repeated. 
Each nuclear site has an independent safety 
engineering group tasked with reviewing events from 
other sites for applicability to their site. Additionally, 
knowledge on event experience is promulgated to 
each site through official NRC documents. However, 
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the result of this regulatory influence is that an 
inquiring and knowledge sharing culture is fostered 
throughout the nuclear industry. This site had an 
excellent knowledge sharing culture and interviews 
and surveys found that engineers were almost as 
likely to capture knowledge because they thought it a 
good idea as they were due to regulatory requirements. 

The organization did not have a formal KM strategy or 
KMS when the case occurred, although by the end of 
the case a formal KM organization had been formed. 
However, the organization did have KMS repositories 
and components although they weren't recognized as 
such. The organization's knowledge was found to 
reside in four major locations: documents, databases, 
your memory, and others' memories. Interviews and 
surveys found several repositories such as email, the 
engineering library, the nuclear design database, the 
work management system, the corporate document 
management system, and the engineers' personal files 
(paper and electronic) and it was determined that this 
constituted the de facto KMS. A few changes in the 
KMS were noted over the course of the case. The most 
significant was a decrease in importance of email, 
which was attributed to changing the email system 
from CCMail to Lotus Notes. The change was 
performed without converting email archives with the 
effect that knowledge was lost. This experience taught 
the organization not to rely on email as a repository. 
Another important change was the reduction in the 
reliance on the "work done" sections of the work 
management system. Cost cutting process changes 
resulted in these sections stored in the Corporate 
Document Management system. 

An important observation on KMS use was that 
amount of use was not a good indicator of the impact 
of KMS use. Several long term organizational 
members during interviews echoed the sentiment that 
it was not how often engineers used the KMS but 
rather that it was the one time that they absolutely had 
to find knowledge or found unexpected knowledge 
that proved the worth of the KMS. An example of this 
was the use of the KMS to capture lessons learned and 
best practices associated with refueling activities. 
These activities occurred on an approximate 18-month 
cycle that is sufficient time to forget what had been 
learned during the last cycle or to have new members 
with no experience taking over these activities. 

Conclusions 

Ultimately, Jennex (2008) found that the organization 
was successful in using the KMS to improve 



organizational performance although no specific 
success measures were identified that could show a 
direct causation between KM and improved 
organizational effectiveness. The next section will re- 
interpret the findings of this case study using the KM 
success definition previously presented. This 
definition and the measures proposed by Jennex, 
Smolnik, and Croasdell (2012) were not available 
when this case study was originally performed. 

Applying The KM Success Definition 

Jennex (2008) found indications that KM improved 
productivity in the nuclear engineering organization. 
These indications included a engineer productivity 
model and external validation based on an overall 
improving trend in capacity factor for the plants. The 
engineer productivity model shown in figure 4 shows 
how knowledgeis applied to the engineer tasks and 
what the outcomes are. Figure 4 will be re-analyzed 
using the KM success measures previously identified. 

The first dimension is impact on business processes. 
There are 6 measures identified: 

1. Improved the efficiency of the supported 
processes 

2. Reduced costs for the supported business process 

3. Positive return on investment for the supported 
processes 

4. Improved the effectiveness of the supported 
processes. 

5. Improved decision making in the supported 
processes 

6. Improved resource allocation in the supported 
process 

Review and inspection of figure 4 leads to the 
conclusion that KM in this engineering organization 
has caused improved process efficiency (using the 
quantitative measures of schedule, number of tasks 
completed, and priorities met); improved process 
effectiveness (using the qualitative measures of 
thoroughness and accuracy); improved decision 
making (client satisfaction measure of decision 
quality); and improved resource allocation (using the 
skill competency measures of task complexity, amount 
of supervision, and correctness). Additionally, Jennex 
(2008) discussed how the engineering organization 
was forced to reduce staff due to deregulation and 
how KM made this possible without the loss of 
organizational capability; resulting in reduced process 
costs. It is concluded that the first KM success 
dimension is met with 5 of the 6 measures being met. 
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The 6th measure was not met as the organization did 
not track investment costs in the initial informal KM 
initiative. 




FIG. 4 ENGINEER PRODUCTIVITY MODEL 

The second dimension is impact on KM Strategy with 
7 measures: 

1. Changes to my organization's KM goals 

2. Creation or modification of knowledge related 
key performance indicators (KPIs) 

3. Changes to the way my organization assessed 
knowledge use in the organization 

4. Changes in my organization's incentives for 
using and sharing knowledge 

5. Increasing its awareness/mapping of knowledge 
sources and users 

6. Increased resources for our KM systems and 
repositories 

7. Creation of new or additional knowledge capture 
processes 

Jennex (2008 and 2008a) discussed what drove 
engineers to capture knowledge and how effective the 
KM tools were, both issues related to the measures of 
impact on KM strategy. Ultimately, it was concluded 
that the amount of use of the KMS was not a measure 
of KM/KMS success, rather it was the intent to use the 
KMS when appropriate that was a better measure. Key 
to intent to use was the perceived benefit model 
proposed by Thompson, Higgins, and Howell (1991) 
and perceived usefulness as described by Davis (1989). 
Perceived benefit and usefulness are both enhanced 
through KM strategy that focuses on ensuring that 
knowledge is captured supporting the knowledge 
users, in this case the engineers. Jennex (2008, 2008a) 
found that there were formal and informal drivers that 



led engineers to capture knowledge. These drivers 
form the basis of the KM strategy. Additionally, 
Jennex (2008) discussed how the organization 
formalized KM by creating a KM position. This 
dimension is met by achieving the measures of 
changing the organization's KM goals, creation of 
knowledge related KPIs, changes to how the 
organization assessed knowledge use in the 
organization, and increasing the resources for KM 
systems and repositories. Additionally, the engineer 
evaluation process was changed during the case study 
with focus more on collaboration and knowledge 
sharing and less on what the specific engineer knew. 
Finally, based on user feedback, the tools used to 
capture, store, search, and access knowledge were 
modified and improved. No data was collected to 
support an evaluation on increased awareness/ 
mapping other than the anecdotal experience of the 
author as the organization became aware of KM 
through the case study and thus becoming more 
aware of where there knowledge was. Then, Jennex 
(2008) provided evidence to support that 6 of the 7 
measures were met and anecdotal evidence from the 
author supported the 7th measure. 

The third dimension is Leadership/Management 
Support with 4 measures: 

1. Increased verbal/political support for KM by 
top management 

2. Increased financial support for KM by top 
management 

3. Increased awareness of KM by top management 

4. Increased use/reliance on KM by top management 

Jennex (2008) discussed how support for doing the 
case study was provided by top engineering 
management through emails sent to each engineer. 
Additionally, the creation of a KM position is evidence 
of increased financial support for KM. Again, 
increased awareness by the Nuclear Regulatory 
Agency, NRC, during the period of the case study in 
addition to the support provided on KM to 
engineering management by the case study author 
increased the awareness of KM within management 
and the formal drivers to capturing and using 
knowledge increased the reliance of management on 
the knowledge repositories. It is concluded that this 
dimension is met and all four measures are supported. 

The fourth dimension is Knowledge Content with 5 
measures (reflected by 8 survey items) 

1. Increased knowledge content in our 
repositories 
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2. Improved knowledge content quality of our 
repositories 

3. Increased use or intention to use of knowledge 
content 

4. Increased identification of needed knowledge 
content and knowledge content sources 

5. Increased demand and/or searching for 
knowledge content 

Jennex (2008) directly measured pratical use and 
found it significant, over 2 hours per day, and 
perceived benefit in the engineering organization and 
found that the engineers would use KM/KMS when 
appropriate. In addition, the previously mentioned 
drivers used to identify knowledge for capture and to 
drive what was captured were all found to be 
considered highly important by the engineers. Finally, 
the use of the engineer productivity model, figure 4, 
by management to assess the quality of engineer work 
resulted in improved knowledge quality and 
increased demand for knowledge content as 
expectations of engineer performance increased. It is 
concluded that this dimension was met and all 5 
measures supported. 

Conclusions 

This paper reanalyzed the data from a longitudinal 
case study of knowledge management in an 
engineering organization in order to demonstrate that 
the dimensions and measures defined in Jennex, 
Smolnik, and Croasdell (2009, 2012) could be used to 
identify and measure KM success in a real 
organization. The longitudinal study originally 
concluded that KM led to an increase in organizational 
and individual productivity but failed to identify any 
measures that could directly link KM to this increase. 
This reanalysis found that 20 of these 22 measures 
were met and could have been used to measure the 
actual performance of KM projects/initiatives in the 
organization. This is significant in that it has 
demonstrated that these measures are those that 
engineering (or any other) organizations should 
identify and use to track KM project/initiative 
performance. The Jennex and Olfman (2006) KM 
success model is also recommended as a model to be 
used to design the KM project/initiative. 

Then, this is significant as it is good management to 
expect that investments in systems and initiatives are 
able to demonstrate that anticipated and/or claimed 
benefits are generated and if the organization 
issuccessful. The economic downturn of 2008 has 



made management of resources critical and 
emphasized the need to demonstrate returns on 
investments and benefits. Prior KM literature has been 
very generic with respect to the measurement of 
knowledge use and KM. It is well known that we 
should re-use knowledge and that doing so should 
make us more effective, efficient, and cost effective. 
Prior to this paper, measures were not available to 
support these claims. 

An organization does not have to show success in all 
the stated measures. Jennex, Smolnik, and Croasdell 
(2012) found that the KM initiative/projects perceived 
to be the most successful had 3.5 dimensions and 17 
measures met while successful projects met 2.25 
dimensions and 12 measures. It is concluded that the 
most successful KM initiative/projects are more aware 
of how to measure success than other KM initiatives/ 
projects. Still, it does mean that not all dimensions and 
measures need to be met; just as this case study did 
meet all 4 dimensions but not all the measures. 
Additionally, there may be additional measures not 
reflected in this paper. It is not expected that there will 
be additional dimensions but that should not preclude 
organizations from considering additional dimensions 
for assessing knowledge use and KM success. 
Repeatedly, organizations should use these dimensions 
and measures to value and demonstrate knowledge 
and KM success and benefits: 

Impact on business processes (6 measures): 

1. Improved the efficiency of the supported 
processes 

2. Reduced costs for the supported business process 

3. Positive return on investment for the supported 
processes 

4. Improved the effectiveness of the supported 
processes. 

5. Improved decision making in the supported 
processes 

6. Improved resource allocation in the supported 
process 

Impact on KM Strategy (7 measures): 

1. Changes to my organization's KM goals 

2. Creation or modification of knowledge related 
key performance indicators (KPIs) 

3. Changes to the way my organization assessed 
knowledge use in the organization 

4. Changes in my organization's incentives for 
using and sharing knowledge 

5. Increasing its awareness/mapping of knowledge 
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sources and users 

6. Increased resources for our KM systems and 
repositories 

7. Creation of new or additional knowledge capture 
processes 

Leadership/Management Support (4 measures): 

1. Increased verbal/political support for KM by 
top management 

2. Increased financial support for KM by top 
management 

3. Increased awareness of KM by top management 

4. Increased use/reliance on KM by top management 

Knowledge Content (5 measures) 

1. Increased knowledge content in our repositories 

2. Improved knowledge content quality of our 
repositories 

3. Increased use or intention to use of knowledge 
content 

4. Increased identification of needed knowledge 
content and knowledge content sources 

5. Increased demand and/or searching for knowledge 
content 
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